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Overview 

# 



When educational historians recall the significant issues of the 
niid-1900s, the advent of educational assessment certainly will be 
remembered, //ow it will be remembered is the real question. 

If these programs get good marks, it will be because they 
contributed in an obvious way to the improvement of education. 
But if assessment is remembered unfavorably, a failure in 
communication — not in assessment itself — will likely be the 
culprit. 

To help avoid the latter fate, this book provides some insights 
into assessment's communication problems and offers suggestions 
for those people responsible for dissemination of test results and 
other phases of assessment and accountability. 

The purpose is not to judge assessment programs individually 
or collectively. Assessment as one component of accountability 
makes good sense. We are much more concerned with the 
communication activities that will make a good assessment 
program acceptable to those affected by it than with the design of 
such programs. 

The quality of assessment communication, to this point, has 
been largely disappointing. Most programs have been given only 
marginal communication support by their originators. Quantity of 
information rather than quality has been the norm. And where 
information exists in quantity, it appears poorly developed, poorly 
timed and based on little — if any — audience research. 

It is well known that people don't accept — much less support 
— programs that are unfamiliar or suspect. Yet assessment-related 
communication from state departments of education and school 
districts to their many publics ranges from embarrassing at worst, 
to promising at best. For example, in one state assessment 
program, teachers are among the last to see the results. How, then, 
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can they be expected to fully appreciate and support the program? 
How can they possibly prepare themselves to explain to parents 
what the numbers in the newspapers mean? 

At the local school district level, the quality of assessment 
communication to parents, staff members, community leaders and 
other important publics has been generally sub-par, even non- 
existent. 

Although we cannot determine what long-term effects assess- 
ment will have on education, it seems clear that the activities of 
today's educational communicator will greatly influence assess- 
ment programs of the future. 

This book is intended to help educators at all levels do a better 
job of communicating with their constituents. There are no 
panaceas. Nor can we guarantee to head off angry citizens before 
they reach the boardroom door. 

What is presented is a look at some good communication 
efforts - programs and practices that have worked at both state 
and local levels to meet the dissemination challenges of educa- 
tional assessment programs and the release of test scores. 



CHAPTER 1. 

Assessment Programs: 
A Perspective 



Although different states and school districts have divergent 
approaches to educational assessment, all assessment programs 
have a common goal — to gather data about the achievement of 
students in schools. Through assessment, states and districts have 
assumed responsibility for assessing the levels of achievement in 
certain basic skill areas and reporting these findings to legislators, 
educators and citizens. The assumption has been that statistics 
help create a better information base to use in making educational 
decisions, and indeed, in funding improved educational programs. 

Individual states have taken divergent positions in the use of 
assessment programs. Some states use their assessment programs 
solely to determine the broad educational needs of children. They 
assess academic areas for educational goals they consider to have 
the highest priority (e.g., reading and mathematics skills). 

Others have programs with overtones of school evaluation. 
These states attempt to judge how effectively the school or school 
district is delivering services to its students and community. 

Most assessment programs have a research component that 
attempts to relate student achievement to other factors such as 
student attitudes, socioeconomic status of the community, class 
size, and drop-out rates, to name a few. 

In many cases, educational assessment programs have been 
irretrievably intertwined with the issue of educational accountabil- 
ity. Accountability programs have become prominent during the 
last decade for a number of reasons: the increasing costs of 
educational services, concern over the inequality of educational 
opportunity for minorities, an urge to increase the schools' 
"efficiency." 

As accountability programs were formulated, the demands for 
data on student achievement and on selected related factors 
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emerged. But, state and federal leaders found little such data 
available. State and district assessment programs were begun, in 
some cases, to provide this missing information. 

The advent of assessment programs was given additional 
impetus by the federal Elementary and Secondary Education Act 
of 1965. Titles I and III of this act made large amounts of money 
available to states and local school districts. Recipients had to 
make a commitment to allocate these funds on a proven "need" 
basis and to evaluate the effectiveness of the expenditures with 
reliable data. (Title III in fact required a needs assessment as a 
basis for fund allocation decisions.) The need for data provided an 
incentive for many states to begin assessment activities. 

Since state assessment programs differ in important ways, it is 
impossible to predict which program will have to face which 
specific obstacles. Yet experience reported across the country 
uncovered a number of common factors that have created serious 
concerns. These are: 

1 . Generation and pubHcation of rankings (percentiles) of 
school and school district scores; 

2. Use of socioeconomic data gathered from students (even 
from anonymous surveys); 

3. Use of student attitude surveys (attitude toward self, 
toward others, toward school, etc); 

4. Use of assessment data to allocate funds to selected 
schools or districts; 

5. Fear of teachers that their performance is being judged by 
the students' scores; 

6. Accusations made by minority groups that the tests are 
biased; 

7. Accusations made by districts that the tests are not valid 
(or are not rehable); and, the list can continue endlessly. 

The following four chapters deal with analyzing the problem 
of disseminating the results of assessment programs, planning such 
a program, implementing the program and finally, of crucial 
importance, the evaluating of these efforts. Twelve basic principles 
on which effective dissemination programs must be based are 
highlighted and discussed. 
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CHAPTER 2. 

Analysis: The First Step 



Dissemination relating to educational assessment programs 
cannot be created in a vacuum. It must b'^ based on analysis and 
help achieve a purpose. It must directly modify, in some way, a 
receiver's knowledge, attitudes, or ability to do some task.* 

State departments of education and school districts should not 
think of communicating educational assessment information to 
the "public" but to various publics. These publics should be rank 
ordered because diffe'ent publics can contribute in different 
degrees to the success of an educational assessment program (or 
because different people have different needs for assessment 
information). Proper consideration of the audiences will help 
assure better understanding of the assessment program through 
successful communication. 

The material presented in the next several pages is partially the 
result of Michigan's involvement in the Cooperative Accountabil- 
ity Project (CAP). Through CAP, the Michigan Dept. of Education 
worked with Erwin Bettinghaus and Gerald Miller of Michigan 
State U. in producing^ Dissemination System for State Account- 
ability Programs, 



Assessment programs tend to be controversial. The 
inadequacy of assessment communication has helped 
illustrate that there is more anger about ideas not 
understood than there is about bad ideas. 



Of course, in some cases there may be valid reason for 
questioning a test's reliability, statistical analysis techniques or a 
particular student questionnaire. And there may be people who 
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simply do not agree philosophically with the use of large scale 
assessment tests in schools. More often than not, however, such 
arguments are simply a human reaction to something not 
completely understood - namely, the objectives and methods of 
implementing an educational assessment program. 



Dissemination must have a purpose. Hoals and objec- 
tives for dissemination must be clearly stated, and the 
"targets" of the messages must be pinpointed. 



In theory, communication takes place whenever a source sends 
some message through some channel to a receiver. This definition, 
however, is incomplete because it omits a critical consideration: 
communication usually has a purpose. The sender wants the 
receiver to react to what has been communicated. In fact, the 
desired outcome is usually some behaviour modification, whether 
a change in awareness, knowledge, attitude or ability to do a task. 

Dissemination, like communication, should be undertaken 
with a purpose. It should help accomplish some end. More 
specifically information disseminated about assessment programs 
should be designed to directly influence groups of people to do 
specific things. Assessment-related dissemination should be per- 
suasive in nature. 



Dissemination is more than telling people about some- 
thing. It is an essential process which uses involvement 
to create a link of understanding between the assess- 
ment program and people it affects. 



Within the context of what assessment is all about - the 
improvement of educational services — the production of persua- 
sive information is both reasonable and desirable. This does not 
mean "news management" or "propaganda." Instead, it means 
state departments of education and school districts should make 
an honest attempt to market their assessment programs. Deliberate 
dissemination efforts should be launched to help people under- 
stand the assessment program, to help them relate to the planning 
function in education, and to help them understand and/or utilize 
assessment results. 

10 
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Dissemination is far more than the wide dispersion of test 
scores (a comparatively simple task). It should be viewed as the 
total process of implementing an innovation in a social situation. 
Dissemination does not begin after the tests have been scored. It 
begins, rather, when an assessment program is first proposed, and 
includes efforts to involve people in the total process from 
beginning to end. If dissemination is not seen this way, the 
problems that accompany lack of commitment to, and support 
for, an informational effort will surface. 



Audiences of an assessment dissemination program are 
diverse and important for different reasons. Critical 
audiences should be identified, prioritized and related to 
the crucial dissemination tasks. 



All too often, those involved in dissemination think of the 
public as one faceless monolith. This is a mistake, for in reality 
there are many publics. An educational assessment program affects 
several distinctly different publics in very different ways . . . and 
each may require a unique communication effort. 

Among the many publics that might be addressed are: 

1. Local school superintendents 

2. Building principals 

3. Teachers of the students tested 

4. All teachers in public schools 

5. Nonteaching school employes 

6. Statewide curriculum groups 

7. Educational associations and unions 

8. College professors ♦ 

9. State board of education 
10. Legislators 

11 
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1 1 . Executive branch of government 

12. Citizen groups 

1 3 . Parent-teacher associations 

14. The press 

15. Advisory councils 

16. State department of education staff 

17. Other governmental agencies 

18. Parents of students tested 

19. Students. 

The list is lengthy but by no means inclusive. It could be 
expanded quite easily, or each public shown could be subdivided. 
For example, most states (not to mention the number of publics 
in a large urban school district) have many professional educator 
associations. Michigan's roster demonstrates the diversity (and 
underlines the challenge facing a disseminator of assessment 
information): 

1 . Michigan Assn. of School Boards 

2. Michigan Assn. of School Administrators 

S.Michigan Assn. for Supervision and Curriculum 
Development 

4. Michigan Elementary School Principals' Assn. 

5. Michigan Secondary School Principals' Assn. 

6. Michigan Reading Assn. 

7. Michigan Council of Teachers of Mathematics 

8. Michigan Assn. of Professors of Educational Administration 

12 
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9. Michigan Educational Research Assn. 



10. Regional associations 

1 1 . Associations in other curriculum areas. 

Such a substantial list raises a very logical question: **Which 
publics should we serve?" The answer is not easy because it relates 
to an understanding of the change process itself. First, a 
determination must be made on which publics are most important 
to the success of the assessment program. Obviously, all are not 
equal. Some have to understand what is happening or the data will 
not be properly collected. Others have to understand what the 
data means, or there will be no impact on school instruction. Still 
others have to believe in what is being done, or there will be no 
funds to do it! The task is to identify the critical publics in some 
order of importance. Then — and only then — can disseminators 
organize their efforts to make sure priority publics get appropriate 
information. 



Different audiences have different informational needs. 
These audiences and needs must be identified and 
accommodated. 



If communication is to be successful — particularly in a 
persuasive sense — the sender must put himself in the place of the 
receiver. This means communication efforts should be designed 
after a proper consideration of the characteristics and informa- 
tional needs of the receivers. 

Obviously, publics will differ, sometimes dramatically, when 
an issue as sensitive as educational assessment is being considered. 
That is why the communicator has to begin by asking a few key 
questions: 

1 . What information does each audience now possess? 

2. Is this information correct? 

3. What attitude does each audience have toward assessment? 

4. Who else is talking to each audience about assessment? 

13 



5. How does each audience feel about the state department 
of education or local school district as a source of 
information? 

6. Does each audience and the state department of educa- 
tion/local school district share the same basic definitions 
of terms? 

To ignore answers to such questions is to guarantee an 
ineffective dissemination effort. 

Depending on the circumstances, it may or may not be 
difficult to gather the information needed to answer these 
questions. A speaker in a workshop situation or an instructor for a 
small college class would find it relatively simple to design a 
written survey or even quiz the audience verbally to assess 
knowledge and feelings. 

In other situations, it may be necessary to gather the data on 
an impromptu basis or to use secondary sources. For example, to 
prepare communications for an audience of school superinten- 
dents who have been involved with the assessment program in 
some detail, a good start might be to review past communication 
efforts for content and timing. Next, contact a few superinten- 
dents to discuss whether their colleagues are likely to understand 
the information you want to communicate to them. Or review the 
superintendents' association publication to see if it reveals the 
membership's attitudes toward assessment. 

In summary there must be a deliberate attempt to learn the 
receivers' current attitudes and knowledge — to become receiver- 
oriented. Whether the effort to gather the necessary information is 
highly structured, moderately intensive, or even hit-and-miss, it's 
obviously better to gather some data than to shoot in the dark. 
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CHAPTERS. 

Planning a 
Dissemination Program 



Without diligent planning and the application of the right 
resources, there cannot be adequate dissemination. And without 
adequate dissemination even the very best educational assessment 
program has no impact. More likely, it will collapse, a victim of its 
own communication weakness. 



6 Determine the dissemination objectives before designing 
the communication activities. 



Start planning dissemination by considering what the purpose 
of communicating with a particular public will be — or, in other 
words, what behavior change is desired in whom. 

Unfortunately, many disseminators never think about behavior 
change. They give speeches, mail letters, issue reports and conduct 
conferences with no idea of what they really want to accomplish. 
If the publics and their needs are varied, which of course they are, 
then what is communicated should depend on which public is 
being addressed. 

In most educational assessment dissemination programs, the 
four basic goals are to: 

1 . Make people aware 

2. Help people be able to cooperate 

3. Convince people to support 

4. Assist people to utilize. 

These goals, of course, apply to different publics in differing 
degrees. Some audiences are important because they help generate 

15 



ERIC 



18 



the data; others because they utilize the results in one way or 
another. For example, it is very important that school principals 
comply with the established procedures for collecting student 
achievement data and returning them to the test processor. On the 
other hand, it is unnecessary for a local Parent Teacher Associa- 
tion to comply with assessment procedures, but it is very 
important that the members be aware of the program. It is not too 
important that superintendents be knowledgeable about the 
details of test construction, but it is vital that they support the 
assessment effort. It is imperative, therefore, to match audiences 
and goals by asking, "What does this audience need to do or know . 
for the assessment program to succeed?" 

The four communication-oriented goals are admittedly broad, 
but they can easily be subdivided into specific objectives for each 
audience. Properly constructed objectives will specify what the 
audience is, what the given conditions are, what behavior is 
expected, and how it will be determined if behavior is changed. 



7 A dissemination program should help build common 
meanings for terms. 



The construction of communication objectives has an interest- 
ing side benefit: it is somewhat sobering because the objectives 
tend to reveal how little is really accomplished, in a behavioral 
sense, by the dissemination steps constructed so laboriously. For 
example, the carefully prepared speech to 100 teachers may really 
boil down to only one objective — prodding those teachers who 
are opinion leaders to support the assessment program. 

But on the very same day the dissemination specialist may face 
another situation requiring a detailed set of communication 
objectives. School principals participating in a workshop on the 
interpretation of report forms must gain a comprehension of what 
the statistical results mean. The construction of objectives 
describing these essential elements will be very helpful in planning 
such workshops. 

Problems of Definition 

One of the difficulties encountered most often in any 
communication situation is the misunderstanding of terms. This 
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problem is frequently brushed aside - assuming the publics all talk 
the same language - but confusion over terms is a real stumbling 
block with assessment programs. 

All words have definitions in the dictionary. Unfortunately, 
many words have several definitions. Shared definitions must be 
established and adequate time must be allowed for them to sink 
in. In fact specific messages must be generated by those 
responsible for interpretation and dissemination solely to establish 
common definitions for key words among fl// the publics. 



Who talks makes a difference. The credibility of 
informational sources is a critical consideration for 
dissemination planners. 



Who talks is critical because different sources generate dif- 
ferent responses. 

For example, there are several potential sources of informa- 
tion about a state assessment program including: 

1. The state department of education as an arm of state 
government; 

2. The chief state school officer or his assistants; 

3. The assessment program supervisor; and, 

4. The individual consultants who work for the program. 

The problem is that all of these sources do not have the same 
credibility. The state department's official pronouncements may 
be received with skepticism by a public that refuses to believe 
"government" can speak with candor and honesty. To a certain 
extent, this reaction will extend to department staff members if 
the public believes they are mouthing the "company line." And 
school districts experience the same problem. 

This phenomenon cannot be ignored. All other considerations 
notwithstanding, the only thing that matters in the final analysis is 
whether the receiver believes the communication is honest. 
Therefore, the communicator must determine what the audience 
thinks of all potential sources as the dissemination steps are 
designed. And sources must be chosen with care. 

An individual acting as a spokesman about assessment should 
be regarded by the audience as competent and trustworthy. This 
should be an important consideration any time a speaker is 
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selected for a particular engagement, any time a letter is sent to 
school superintendents, principals or parents and any time 
someone has to answer a telephoned question. Obviously, the 
assessment program will be ahead of the game if its director is 
perceived as competent and trustworthy. 



9 Involvement of key people and groups heightens com- 
mitment. 



It is not necessarily possible to communicate everything to 
everyone. Instead, specific messages should be sent to different 
publics for different purposes. While all audiences are not equally 
vital to the success of the assessment program, some people must 
be supportive or nothing will get done. 

It is all too common when introducing new programs to pay 
attention only to those who are formal approvers — persons with 
the power within a bureaucracy to allow a program to exist or not 
to exist. Yet the informal approvers - a key university professor, a 
respected superintendent, or, in a given school district, key 
teachers - are often even more important. As powerful opinion 
leaders, they can aid or handicap an assessment dissemination 
program. They cannot be ignored. 

Key opinion leaders are likely to support an assessment 
program if they are involved. Besides, their involvement gives a 
department of education or school district the chance to get 
professional advice and creates a feedback loop to help judge the 
effectiveness of the program. 

Involvement can appear at many levels and in many forms. 
Here, for example, are a few suggestions for state departments 
which with adaptations could help school districts: 

1 . Establish an advisory councU composed of ten to fifteen 
key persons to meet monthly with assessment staff to hear 
about and react to program plans. 

2. Establish a technical advisory group of three to five people 
from colleges and local school districts to help with tech- 
nical problems. 

3. Use local districts to help develop learner objectives and 
test items for use in the assessment program. 
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4. Establish a research support unit to help college professors 
and students use the data being collected. 



5. Involve state and regional level curriculum groups in the 
interpretation of data generated by the program. 

6. Include local school personnel in presentations made to 
educators about the assessment program. 

Some disseminators avoid involving other people because: (1) 
it takes effort; (2) they do not want the "family skeletons" to be 
revealed; and/or (3) they do not want the advisors to think they 
are being given formal approval power. 

Involvement, of course, does take effort, but it is well worth 
it. Family skeletons tend to be revealed to a certain extent, but 
the old adage about "honesty pays" still holds. There is no reason 
why advisors cannot live with the fact that they are not final 
decisionmakers as long as they are told this at the beginning and 
reminded once in a while throughout their period of involvement. 



Assessment program disseminators should establish an 
early working relationship with public relations staff 
members or consultants. 



Most state departments of education and an increasing number 
of school districts have a public information office that specializes 
in communication, distributes news releases and produces a wide 
variety of publications. Such an office can be a significant asset in 
the dissemination effort. Yet it should be remembered that 
dissemination (as defined in this book) involves more than just 
reaching publics through the mass media. Therefore, it is best for 
those responsible for assessment program dissemination to estab- 
lish early a working relationship with the information office. 
Answering key questions, such as the ones following, can help 
assure a cooperative effort by clearly establishing ground rules. 

1. Who can distribute an assessment press release? 

2. Who should compose the assessment press release? 

3. Who should approve an assessment press release? 
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4. Who can authorize the composition and publication of an 
assessment report? 

5. Who can authorize the creation and distribution of 
audiovisual materials relevant to the assessment program? 

6. Who controls the speakers' bureau on assessment affairs? 

7. Who has authority to call a press conference to discuss 
assessment results? 

The answers should help establish a positive working relation- 
ship between the assessment program staff and the public 
information office. 



Effective dissemination requires adequate human and finan- 
cial resources. 



Dissemination is an important part of any assessment program. 
It must be given attention and it must be given adequate resources 
— time, money, materials, and people. 

To a certain extent, most staff members are involved in 
dissemination whenever they answer a letter, return a telephone 
call or make a speech. But there is a need in a state education 
department and most large school districts for a staff person to be 
ultimately responsible for all dissemination activities. A good 
beginning point might be one dissemination person for every four 
assessment staff members. 

Looking at the potential scope of a statewide effort, it has to 
be recognized that particular state characteristics have a bearing on 
the size of the dissemination staff and its operating style. Hawaii, 
as a one-district state, would operate much differently from 
Michigan with almost 600 districts. The state assessment program 
staff cannot be expected to contact directly all of the state's 
teachers, principals or superintendents. They must, instead, train 
intermediate or regional-level people who can, in turn, train 
others. 

The train-the-trainer method is well known and has been 
applied in many different situations. An assessment coordinator in 
each local district can serve as the link between the state 
department and the local school staff. The coordinator would 
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receive the various publications and attend workshops to become 
trained in assessment matters. Or else an intermediate district staff 
person, if an intermediate school district or regional school office 
exists, can be trained to assist the local assessment coordinators. 

Although this method has the advantage of extending the 
capability of the assessment staff to reach large numbers of 
people, it risks introducing error into the system. The trickle-down 
effect can apply to the extent that the integrity of the information 
can be assured through, perhaps, two levels. Beyond that things 
begin to get a little muddled. 

This can be avoided to some degree by producing printed 
and/or audiovisual materials for the local or regional person to 
use in training sessions. This will at least help preserve the 
accuracy of basic facts. 

There is no fixed guideline to use for dissemination expendi- 
tures. It depends upon the state, the length of time the program 
has been in operation, its goals and its complexity. A reasonable 
figure for an ongoing assessment program, not including personnel 
costs, might be 10% to 15% of the program's budget, exclusive of 
developmental costs. So, if it costs $300,000 to print tests, score 
answer sheets, generate printouts, ship materials, and produce 
statistical summaries, it would be reasonable to spend from 
$30,000 to $45,000 more on reports, AV materials, letters, travel, 
etc. The overall expenditure is a trade-off; if you spend less 'than 
necessary, you will sacrifice effectiveness. 

The very best tests, the most accurate scoring and analysis, and 
the most efficient method of packing and shipping will mean 
nothing if the assessment program is not understood, appreciated, 
and utilized - and on schedule. These are all dissemination-re- 
lated, but their importance is all too often ignored when resources 
are allocated. 

In summary, when planning a dissemination program, atten- 
tion should be focused on: 

1. Building communication objectives that clarify what spe- 
cific effect is desired for each particular audience; 

2. Helping audiences understand what assessment means 
(because words mean different things to different people); 

3. Selecting trusted, well-known sources to originate mes- 
sages; 
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4. Involving key people and groups to increase the likelihood 
of their commitment; 

5. Building good relationships with the public information 
office; 

6. Providing adequate human and financial resources to 
support the dissemination efforts. 
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CHAPTER 4. 



The Communication Phase: 
Plans Into Action 



The general principles of dissemination discussed in the 
previous pages were well tested in 1973 when the Michigan 
Educational Assessment Program (MEAP) switched from norm- 
referenced to objective-referenced tests. The new program was a 
dramatic departure from the old one and dictated that various 
publics acquire new skills, attitudes, and understanding. FIGURE 
1 on page 25 shows some of the contrasts between the new 
program and the one used from 1969 through 1973. 

The dissemination staff generally operated under the four 
broad goals previously outlined: 

• To raise awareness; 

• To elicit support; 

• To obtain compliance; 

• To encourage utilization. 

The primary audience for Michigan's dissemination efforts was the 
local school district staffs. These were the people who would be 
collecting the data and utilizing the feedback. 

Existing Preliminary Structure 

The 1973 dissemination efforts did not begin in a vacuum. 
Disseminators built on an existing structure and were able to 
capitalize on the products of previous years, including: 

• An advisory council that had met approximately monthly. 
Membership background was varied. 
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• An existing structure of publications: Objectives and 
Procedures (outlined program plans); Individual Student 
Report (interpretation of public data); School and District 
Report (interpretation of school and district data); Techni- 
cal Report; Local District Results (listing of scores for each 
district). 

• Each district had been required to appoint a local 
assessment coordinator to implement the collection of 
data and reporting of results. 

• Routine workshops for local district administrators had 
been held at least twice each year across the state, 

• The state's curriculum specialists in reading and mathe- 
matics had been involved in constructing the performance 
objectives. 

• Several local districts had been involved in constructing 
and validating the items that comprised the new tests. 

And, although not necessarily a welcome fact, the Michigan 
assessment program had gained considerable attention during the 
past few years. It seemed everyone was interested in discovering 
how (or whether) the new tests would work. 



Michigan's 1973 Assessment Dissemination Program 

The assessment staff designed the dissemination program to 
give people enough time to become aware of the changes in the 
program and to learn the new vocabular>^ of objective-referenced 
tests. The local assessment coordinators trained local teachers with 
aids provided by the Michigan Dept. of Education. The dissemina- 
tion effort was also aimed at teachers not directly involved in the 
testing and at parents of the students tested. These efforts did not 
stop with the collection of data. They extended, in fact, through 
the utilization phases of the program. 

The dissemination plan adopted by the Michigan assessment 
staff called for a variety of activities. To help the reader benefit 
from some of the ideas tried in Michigan, they have been listed 
below with a brief explanation: 
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FIGURE 1 

Revised Michigan Educational Assessment Program 



Old Program New Program 

(1969-1973) (1973-1974) 



Content: 

Word Relationships 

Mathematics 

Reading 

Mechanics of Written English 

Tests: 

Norm -referenced 
Timed 

Scoring Systems: 
T-scores 

Reporting Systems: 

Student means & percentiles 
School means &'percentiles 
District means & percentiles 

Grades Tested: 
Four & Seven 

Testing Date: 
January 



Content: 

Word Relationships 

Mathematics 

Reading 

Tests: 

Objective-referenced 
Untimed 

Scoring Systems: 

Attainment of objectives 
T-scores (Word Relationships only) 

Reporting Systems: 

Percent attainment for objectives 
for students, schools, districts, 
state 

Student, school, & district means and 
percentiles for Word Relationships 

Grades Tested: 
Four & Seven 

Testing Date: 
Late September 
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1. A one-page letter was sent to all superintendents from the 
state superintendent alerting them to forthcoming changes 
in the program. 

2. A letter was sent to superintendents and local coordinators 
from the department's research director with an attach- 
ment describing the new program in brief. Recipients of 
the letter were encouraged to reproduce the attachment 
and share it with building principals and teachers. 

3. An eight-minute color, narrated filmstrip was developed 
for local administrators to use with all teachers in early 
September. The filmstrip gave a general introduction to 
the new program. Copies were distributed to each school 
district. 

4. A 4- X 8-inch information card was printed and provided 
with the filmstrip for distribution to all teachers. This card 
summarized the filmstrip content, 

5. Several news releases were sent out and articles were 
printed in state education media. 

6. Staff members made countless presentations and speeches 
about the new tests to various educator groups across the 
state. 

7. Briefings were held for local assessment coordinators in 
early September. 

8. On test day, take-home cards (4- x 6-inch) which capsul- 
ized the purpose of the assessment program were provided 
to each local district. Students were to give the cards to 
their parents on the day the tests were administered. 

9. An 18-minute color, narrated filmstrip was developed for 
use with fourth- and seventh-grade teachers to instruct 
them in how to interpret the test results. Copies were 
distributed to all school districts. 

10. In-depth interpretation workshops were held in November 
across the state. These sessions were conducted by selected 
local educators and department staff members. They were 
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attended by key local staff members (in most cases a 
curriculum coordinator, key principal or selected teachers) 
chosen by each district superintendent. 

1 1 . "Brief-folios" were produced for the workshop partici- 
pants. These informational packets included detailed mate- 
rial on interpretation of data as well as other handouts. 

12. Several communications were directed to the state Board 
of Education, including use of the filmstrip described in 
(3) above. 

13. "Regular" publications were developed and distributed. 
An exception was the Local District Results, which was 
replaced by a new report titled Statewide Summary. No 
comprehensive district-by-district listing of scores was 
published by the state. 

14. Contacts were established with Michigan reading and 
mathematics curriculum groups to help the department 
interpret the statewide test data and critique the test items 
and objectives. 

15. Statewide results were communicated to state legislators 
by the department's legislative aide. 

16. Data were made available to researchers across the nation 
for use in educational studies. 

17. Briefings were conducted for Department of Education 
staff members not involved in the assessment program. 

18. Innumerable individual telephone contacts, correspon- 
dence and on-site consultations were conducted by staff 
members. 

The number of communication activities dictated that the 
dissemination program be carefully timed. Consequently, a de- 
tailed time-line was developed to show the beginning and ending 
dates of each activity (see Figure II, page 29). The importance of 
such a planning chart should not be underestimated. The 
dissemination steps must take place in a logical sequence, and 
must be related to other phases of the assessment program. The 
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two key points in the illustration are the test administration dates 
and the date of receipt of test results by the local districts. The 
major dissemination tasks revolved around these two dates. 



Resources Required for 
Michigan's Dissemination Program 

The 18 dissemination steps used in Michigan cannot easily be 
assigned dollar costs. In some cases, the tasks were included in the 
regular responsibilities of a staff member. In other cases, an 
outside consultant was hired. Costs have been outlined where 
possible. 

1 . Program Reports. The basic program reports were composed 
by staff members and printed by either the department or 
the test support contractor. Some were distributed by the 
department; others by the test support contractor with the 
test materials. 

2. Magazine articles. Articles and newsletter announcements 
were composed by staff members as part of their regular 
duties. 

3. Filmstrips. A master negative of a color filmstrip with 135 
frames was commercially produced for about $3,500 
(including the narration). Packaging, mailing, and duplica- 
tion costs were about $4.50 per set. 

4. Test-day Take-honie Cards. The test-day take-home cards 
were printed on 65# cover stock, 4x6 inches. They were 
composed and printed commercially for $2,500 (320,000 
copies). 

5. Teacher-Cards. The 4- x 8-inch teacher cards to accompany 
. the filmstrip were printed commercially on 65# cover 

stock for approximately $ 1 ,600 (100,000 copies). 

6. Workshop Brief-Folios (information kits). The assembly 
and printing of workshop materials (composed by depart- 
ment stafO were done commercially for about $500 
(1,000 copies). This packet contained about 15 sheets, 
printed on both sides, and glued on one edge. 

28 



ERIC 



31 



< 























1 1 







:3 
to 



o 

(0 

c 

1 

in 

5 

O 

c 
E 



o 



3 
< 



1 



1 1 



< 



'S 

1) w — . 
O c o 
o u o 



o 
a 

o 



3 O 
T3 O 



oti: 

H en 



a 

•s 

1/3 

IS 



5 « 



0 O 

a 3 

1 2 



O 

a 

a 
o 

o 



•a 



cu cu ft* 



I 1 



o o 



•a 

o 

u 

13 
55 



•a o 

I E 
H a: 



vo 



00 a\ o oi f<i 



29 



ERIC 



32 



7. Workshops. The department staff conducted seven brief- 
ings and eight workshops for assessment coordinators. 
Costs were minimal, consisting only of the usual travel 
expenses. 

8. Utilization Studies. Two contracts for $1,500 each were 
awarded for the interpretation of assessment data to two 
statewide curriculum groups. The $1,500 did not com- 
pletely cover their costs. Both organizations had to rely 
upon membership generosity to help offset the actual costs 
of producing the analysis. 

9. Staff Resources. In 1973, the Michigan Educational 
Assessment Program staff included a director, three consul- 
tants, one analyst and three secretaries. One consultant 
was assigned dissemination duties and was assisted by the 
other persons as necessary. (The staff has since been 
expanded.) 

ResuIt-of-Experience Ideas 

The dissemination program undertaken by the Michigan 
Educational Assessment Program in 1973 was effective in many 
ways. Educators, for example, were well aware of the changes in 
the program and completed the tasks required of them. Some 
educators followed department suggestions and tried to weave the 
test data into their own testing programs and use the data to 
improve instruction. Other school officials, unfortunately, did not, 
and, in fact, limited MEAP's impact by restricting use of the data. 

A review of the process showed that dissemination to 
legislators, the State Board of Education and parents was 
inefficient. It also became apparent that improvement was needed 
in the reporting procedures to teachers whose students were 
tested. Appropriate modifications were made in 1974, and further 
changes are planned for the future. Some of the following 
result-of-experience ideas are being added to the Michigan dissemi- 
nation effort: 

• A "test results folder" can be developed and used to 
transmit data to teachers about their students and class- 
rooms. This folder, attractively composed and printed, 
should contain essential information about how to read 
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the data and utilize it. The folder would eliminate the need 
for a teacher havmg to resort to reading a booklet every 
time he or she wishes to review the data, 

• Different types of reports of the statewide data can be 
prepared for the three most important audiences of the 
assessment program - the legislators, the state board of 
education, and the local district administrators. The first 
two reports should be brief and nontechnical with great 
use of visual displays of the data, 

• Specialized report summaries should be prepared for 
teachers to use with parents. The usual computer printed 
format given to teachers is too imposing for parents, A 
simple summary would be preferred. 

• Special efforts can be made to communicate with univer- 
sity teachers about the assessment program. These people 
can benefit from the data in their research efforts. 
Generally, they have not been fully aware of what data 
existed. 

• Attempts should also be made to make instructors in 
colleges of education knowledgeable about the assessment 
program so they can communicate accurate information to 
their students. It is not unusual to find a misinformed 
college teacher passing along erroneous information about 
the assessment program to hundreds of pubHc school 
teachers and administrators in a year. 

• The state department of education should prepare training 
materials and workshops to help local administrators learn 
how to release test data to various publics. Most adminis- 
trators are not knowledgeable in this area and do not know 
how to make their test data understandable to parents - 
nor do they necessarily know how to help teachers use the 
data properly, 

• State department of education personnel should attempt 
to use local school district personnel in training sessions. 
Such people might even serve as authors of reports or 
special interpretations of the assessment data. These 
activities would help get people "involved" and their 
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contributions generally will increase the department's 
credibility. For example, instead of having department 
personnel discuss statewide math test results with the 
press, it would be better to have the president of the state 
mathematics association do it. 

Of course, there are many other suggestions that would 
improve a given dissemination effort. Perhaps the ones listed above 
will serve as mind ticklers so the reader can generate others. 

A good dissemination program does not concern itself 
solely with the reporting of test data. It includes all the 
steps taken to involve people, to inform them, and to help 
assure that the test data is used to improve educational 
programs. 
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CHAPTER 5. 

Evaluation 



Effective dissemination activities need strong evaluation 
components. 



Throughout this book, the need for varied dissemination 
approaches for different types of audiences to meet varied 
communication objectives is stressed. And continuous evaluations 
are necessary to determine whether or not the objectives were met 
for the particular audience. 

The use of feedback (evaluations) in a communication setting 
should not be a new concept for anyone. Everyone uses feedback 
daily as they talk to their friends and coworkers. Facial expres- 
sions or the tone of voice allow the listener and speaker to 
determine subtleties in a conversation. Ignoring such feedback 
guarantees ineffective communication. 

Advertising firms constantly use evaluations to determine the 
effect of their campaigns. Most persons have received a phone call 
from a survey company wanting to know how they felt about a 
particular product or advertisement. 

Evaluation of communication is not something strange or new. 
Yet educators often act as if they don't need it. Speeches are 
given, workshops are conducted, and thousands of reports are 
written without ever asking the key question, "Is this effort 
accomplishing the objective?" 

In fact, to make matters actually worse, far too many 
evaluations focus solely on how many jniessages were sent. This 
means that a "good" dissemination campaign is one that generates 
a lot of newspaper clippings and several thick reports. Such 
"evaluations" are worthless. Forget the volume of messages - 
concentrate instead on the effects on the receivers. 
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Educational evaluators today talk about "formative" and 
"summative" evaluations. In the simplest terms, the former refers 
to the gathering of data to modify, if necessary, a program as it is 
bemg developed. The latter designates a judgment about the final 
effects of a program. Both approaches can be used in the 
evaluation of dissemination programs. 



Formative Evaluation 

Most writers revise and edit a manuscript several times 
themselves before submitting it to others for their comments and 
revisions. This is formative evaluation in a very basic sense. Or, as 
another example, consider the state assessment staff that plans a 
workshop for local school principals. If the staff members are wise, 
they will at least discuss their plans and materials with some key 
advisors, or, better still, actually conduct a try-out session for a 
small group of principals. More information can be obtained by 
asking workshop participants to complete a survey immediately 
following the sessions. Such feedback can help identify problem 
areas that were not detected during preparation. 

It is wise to gather input from others while developing each 
activity in a dissemmation program. The input is particularly 
valuable when it comes from a sampUng of the audience you are 
trying to reach. Depending on circumstances, it may be advisable 
to repeat this cycle several times. 



Summative Evaluation 

An evaluation at the end of a particular dissemination activity 
or program should determine whether specified objectives were 
attained. For example, if the program was designed to help local 
district assessment coordinators comply with required'tasks, there 
should be some means of determining whether they did. This 
could involve monitoring test administration in several local 
schools, tabulating errors detected by the test processing com- 
pany, or surveying the coordinators to determine if they felt they 
were adequately prepared. Each of these three approaches has its 
merits and Habilities. The important thing is to try to learn 
whether objectives were attained. 
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In a larger context, a complete dissemination campaign might 
be directed at school superintendents (or some other public) over 
a long period of time to gain support for the assessment program. 
In this case, a statewide public opmion survey utilizing personal 
interviews of several hundred people could be taken. While this 
method can be quite expensive, it yields data that can be 
invaluable. 

A less expensive method would be to utilize a doctoral student 
or staff person to conduct a mail survey. Such surveys can be 
completed for mmimal costs; e.g., paper, postage, and processing. 

The Michigan Dept. of Education found all three methods to 
be useful. Samples are featured on pp. 36-39. Also mcluded is an 
evaluation form used to survey parents in the Madison, Wis., 
schools on their knowledge and opinions of testing in the district. 
The form was the inside back cover of the February 1974 issue of 
the school district's publication, The Learning Tree. The issue 
focused on testing and evaluation. (See p. 40.) 



Summary 

Dissemination programs should be evaluated as they are*being 
devised, as they are implemented, and when they have been 
completed. Evaluations should be done in terms of the original 
objectives to determine whether the desired behavioral changes 
were accomplished for the audience in question. Without such 
evaluations, one can only guess about effectiveness or merely 
count messages. Neither is appropriate. 
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RA-3087 Michigan Department of Education 

4/74 Research, Evaluation and Assessment Services 

MICHIGAN EDUCATIONAL ASSESSMENT PROGRAM (MEAP) 

Box 420, Lansing, Michigan 48902 

1973-74 MEAP SUPERINTENDENrS FEEDBACK SURVEY 



1. Please indicate the total enrollment of your school district by checking the appropri- 
ate box. 

a. 010,000 and above b. 04,500 to 9,999 c. Dbelow 4,500 

Each of the following statements or questions pertains to the 1973-74 Michigan Educa- 
tional Assessment Program. Please CHECK a box below each statement which best 
describes the DEGREE to which you feel the statement applies to your district. 

2. The 1973-74 objective-referenced test results are MORE USEFUL than those of the 
past four years. 

□ □ □ □ □ 
1 2 3 4 5 

less useful more useful 

3. The objectives TESTED are generally ACCEPTABLE in my district. 

□ □ □ □ □ 
1 2 3 4 5 

acceptable not acceptable 

4. I am generally SUPPORTIVE of the new objective-referenced tests used by the 
MEAP. 

□ □ □ □ □ 
1 2 3 4 5 

not supportive supportive 

5. My school board is generally SUPPORTIVE of the new objective-referenced tests 
used by the MEAP. 

□ □ □ □ □ 
12 3 4 5 

not supportive supportive 

6. My fourth and seventh grade teachers generally regard the objective-referenced In- 
dividual pupil results in Reading and Mathematics as a VALUABLE tool. 

□ □ □ □ □ 
1 2 3^4 5 

veryvaluable of no value 
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7. My fourth and seventh grade teachers generally regard the individual pupil results on 
Word Relationships as a VALUABLE tool. 

* ^ □ □ □ □ □ 

1 2 3 4 5 

very valuable of no value 



8. The district and school results on Word Relationships are VALUABLE tools. 

□ □ □ □ □ 

1 2 3 4 5 ' 

of no value very valuable 

9. Do you support the inclusion of the Word Relationships test in future assessment 
batteries? 

(Check ALL that apply.) 

a. □ NO 

b. □ YES, at fourth grade ONLY 

c. □ YES, at seventh grade ONLY 

d. □ YES, at local district OPTION 

e. □ YES, on a pupil SAMPLING basis 

10. The 8-minute filmstrip, "Michigan's New Teacher Aid," was designed to present gerv 
era! information about the MEAP to all teachers. Please estimate the PERCENT OF 
TEACHERS in your district who SAW this filmstrip. 

a. 00-25% b. 026-50% c. □51-75% d. 076-100% 

11. Did you feel this filmstrip, "Michigan's New Teacher Aid," was helpful to your staff? 

a. Dyes b. Dno 

12. The 18-minute filmstrip, "Interpreting the Results of the 1973-74 MEAP," was 
designed to provide specific information about the MEAP tests to fourth and seventi. 
grade teachers. Please estimate the PERCENT of FOURTH and SEVENTH GRADE 
TEACHERS in your district who SAW this filmstrip. 

a. 00-25% b. 026-50% c. 051-75% d. 076-100% 

13. Did you feel this filmstrip, "Interpreting the Results of the 1973-74 MEAP," was 
helpful to your staff? 

a. OyeS b.ONO 
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RA-3079 Michigan Dtptrtrrunt of Education 

1/74 RESEARCH, EVALUATION AND 

ASSESSMENT SERVICES 
Michigan Educational Assessment Program 
Box 420, Lansing, Michigan 48902 

ASSESSMENT COORDINATOR'S FEEDBACK SURVEY 



1. Please indicate by CHECKING the appropriate box the total ENROLLMENT of 
your school district. 

□ a. 10,000 and above 

□ b. 4,500 to 9,999 

□ c. below 4,500 

2. Please CIRCLE the number below which best describes the DEGREE to which you 
feel you were informed through the Michigan Educational Assessment Program 
workshops, manuals, magazine articles, etc. about the changes in the Program and 
the new test administration procedures. 



1 2 3 4 5 6 7 

Inadequate Adequate 

3. Please indicate in the space below any particular educational assessment information 
NEEDS you had which were not met by the Department's efforts. 



4. Did you receive sufficient quantities of the rr«>terials to properly administer the 
assessment tests? 

□ YES 

□ NO 

5. If you answered "No" to the previous question, were you able to obtain the necessary 
materials from the. intermediate school district offices? 

□ a. Yes; all I needed 
D b. Some of them 
D c. None 




ERIC 



41 



TEACHER ATTITUDES 
1974-75 MICHIGAN EDUCATIONAL ASSESSMENT PROGRAM 
TEST RESULTS 

A Paper Submitted to the Staff of the 
Michigan Educational Assessment Program 
by Anthony Aquino 



1 . Which category best describes the location of your school? 

Tri-county Metropolitan Area (Wayne, Oakland and Macomb 
counties) 

Lower Peninsula, excluding Tri-county Metropolitan Area 
Upper Peninsula 

2. Which instructional area do you have as primary teaching responsibility? 
(Check both if responsibility is equal.) 

Reading-English 
Mathematics 

3. Which assessment test reports have you received from the testing session 
conducted in the fall of 1974? (Check ALL that apply.) 

Individual Student Report (Computer Printout) 

Classroom Listing Report (Computer Printout) 

Classroom Summary Report (Computer Printout) 

Individual Student and Classroom Reports - Explanatory 

Materials 

None of the Above 

4. When did you receive the MAJORITY of these reports listed in Item 3? 
(Check ONE only.) 

November 1974 February 1975 

December 1974 March 1975 

January 1975 

I have not received any of these materials. 

5. Have you studied the assessment test results for your students? 

Yes 
No 

6. How well prepared are you to make use of the assessment test results? 

Very well prepared 

Well prepared 

Not very well prepared 

7. Do you have access to the total set of state minimal objectives for 
mathematics and/or communication skills^l^ther your own copy or a 
building copy)? (Check the appropriate bgf %a. and b.) 

Mathematics 
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PARENTS ARE ASKED 
TO RESPOND HERE 



to (tetemm what you know about testing 
and if you want any changes 

The following survey is being taken to determine what parents know about their child's school 
testing program and what changes, if any, they feel should be made in the evaluation process. Fill in 
the questionnaire and send It to: 

Public Information Office 

Testing Survey 

Madison Public Schools 

545 West Dayton 

Madison, WI 53703 

The results will be discussed in a future issue of The Learning Tree. 

CUP AND MAiL 

Please Circle or Check Your Responses 

1. What type of testing program is used at your child*s school? 

A. Criterion-Reference 

B. Traditional Examination 

C. Combination 

2. Has your child ever been involved with any other type of testing program? Yes No 

What method? 

3. Has your child taken standardized tests given by the school? ' 

No 

4. What do you expect a standardized test to reveal? 

A. How much you child knows. 

B. How well your child can learn. 

C. How well your child is doing compared to the other children. 

D. Other \ 

5. What do you expect a classroom test to reveal? 

A. How much your child knows. 

B. How well your child can learn. 

C. How well your child is doing compared to the other children. 

D. Other 

6. Do you think that there should be more/less standardized testing?^ 



7. Do you think classroom exams should test: A. Content knowledge B. Concepts C. Combination 

8. Are you satisfied with your child's testing program at his school? Yes No 

9. In what ways do you think your child's testing program can be improved ?^ 



10. What school docs your child attend? 
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CHAPTER 6. 



Regional Training: 
One State's Stoiy 



Note: The following chapter is a series of recollec- 
tions, impressions, and assessments by the authors of 
their experiences during the first-time release of 
statewide test scores in Maryland. It is written in that 
style and reflects their personal as well as objective 
views of this experience. 



After the first man ate an oyster and lived, those who followed 
were able to go through the experience with substantially less 
trauma. If assessment can be related to an oyster, we (the 
Maryland State Dept. of Ed.) ate the oyster. 

Quite candidly, those associated with the Maryland Dept. of 
Education in 1974 approached the first-time release of test results 
in panic. This decidedly negative attitude largely resulted from 
Maryland's accountability program being initiated under a legisla- 
tive gun. The state's education establishment had had little to say 
about the original legislation. 

Although the law provided two years of lead time, it soon 
became apparent that development of a full-blown accountability 
program would take longer than the time allowed. When the 
deadline for assessment neared, there was little choice. During the 
first year assessment would mean standardized tests. 

The department proceeded on the assumption that scores 
would be low where they ought to be high, and lower still where 
they ought to be just low, and that some people would not be 
satisfied no matter what the results. 

There were three ways — all chilling — in which the data could 
have been presented; the statewide summary, the district-by-dis- 
trict report, and the school-by-school report. The most detailed 
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report has the deepest pitfalls. Invidious comparisons (**best 
school in the state," "worst school in the state") would result in 
media comment. Other potential nightmares included demands by 
many of parents that their children be transferred from this 
"dumb" school to that "smart" one, or that this principal or that 
superintendent seek other employment. 

Under circumstances like these, panic was entirely rational. 
But state agencies have leadership responsibilities, and leadership is 
what they must offer. Here's how Maryland's Dept. of Education 
worked regionally to help local district representatives understand 
their test results and prepare for dissemination: 

The single most effective piece of advice a state education 
official can offer local school systems is that they release 
their school test scores as soon after they get them as 
possible ... and before the state department of education 
does it. 

The payoffs for this strategy are manifold: 

• Random advance release by local school systems reduces 
the likelihood of system-by-system comparison. 

• Advance release gives local school districts the advantage 
of initiative. Waiting for statewide release puts the district 
in the uncomfortable position of simply reacting to 
questions - questions that can often amplify trivia and 
ignore substance. 

• Local system release helps develop a greater sophistication 
among reporters and the public about test results. It also 
reduces the likelihood of public panic when the state 
report comes out. 

Local release places a heavy burden on principals to explain 
the test results. Unfortunately, some districts just cut them loose 
to make the best of it, and it's no wonder most principals were 
petrified by the prospect. In all cases principals are entitled to all 
the support they can get. 

The Maryland State Dept. of Education held a series of 
regional workshops, each just half a day long. A limited number of 
principals from each local school system - at least one represent- 
ing elementary, middle, and secondary schools - were invited. 

42 

45 



ERIC 



Central office administrators were involved in the program and the 
information specialists of the districts that had one also took part. 

Participants were told what, when and how test results would 
be released by the State Department of Education. This provided 
background and served to stimulate interest in the workshop. 

An outside consultant^ discussed techniques for the local 
release of test results. The presentation centered on the public 
relations aspects of assessment dissemination. Participants were 
given insight into what the public knew about the program, what 
types of information they would want, and the best channels of 
communication for delivering the information. 

The consultant also used the small group approach to help 
participants identify key publics in their local districts and the 
most promising communication techniques for reaching each. 

Advance copies of district test scores were distributed for 
review by the participants. A member of the National School 
Public Relations Assn. (NSPRA) worked with each group (dis- 
trict). Potential problem areas were identified and evaluated from 
a communicator's point of view. 

Using their own test scores, participants worked to isolate the 
messages that would go to key audiences. The messages, channel 
of communication and timeline were reported district-by-district 
to the entire audience. (This was feasible because only 5-10 
districts were represented at each workshop.) The session ended 
with questions on both communication and testing mechanics. 

For this process to work, the participants had to go home and 
share their new-found expertise with their colleagues. As they 
were dismissed, they were urged to repeat this program at staff 
meetings back home. 

One of the insights gained from experience was that principals 
are often unfamiliar with the "numbers game" called test data 
I interpretation. 



Beware of Statisticians 

The natural impulse in attacking such a problem is to assemble 
the test specialists and statisticians to explain. But beware of this 
seemingly simple approach. Statisticians and test specialists enjoy 
talking to one another, and they do that very well. But they have 
trouble with educators. The evidence: Everything goes well until 
somebody asks a question. It's all downhill from there. 

If you have a test specialist or statistician on your staff who 
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can popularize the presentation, you are in proximity to a rare 
jewel. If not, have them work very closely with your information 
specialists as they prepare their explanations. 

A local districtwide release, on the heels of local school 
release, requires planning. Local districts with public information 
specialists are well equipped to handle that planning. In Maryland, 
local public information specialists offered a hand to school 
systems lacking such expertise. 

Some local superintendents seek "standard" fill-in-the-blanks 
news releases. Reflection suggests that is not a particularly good 
idea. It carries with it a degree of news orchestration that is 
inconsistent with developing public confidence. More importantly, 
standardized releases prevent the tailoring of local information to 
local audiences. But the development of a statewide fact sheet is 
very much in order. How many kids were tested, in which grades, 
in which subject? When? At what cost? For what purpose? 
Statewide uniformity in reporting basic information of this type 
suggests good communications, not good collusion. 

One serious problem in dealing with media is that test results 
yield only performance data. Reporters will, of course, take a bead 
on the weakest performance areas and ask, "Why?" Educators will 
be inclined to speculate defensively, or to philosophize on the 
limitations of standardized tests, which comes down to the same 
thing. 

Move carefully. It won't do to say you don't know what the 
results mean or why they are what they are. On the other hand, 
you risk the charge of "ALIBI" in a big headline if you fill the air 
with explanations. Try to limit explanations or comments on the 
results to statements you can document - and be ready to supply 
the documentation. 

The Message 

If your test results sag in places, your message needs to contain 
the same information, whether you are a principal, a local 
superintendent, or a state superintendent: 

1 . These are the test results 

2. Here is where we did well 

3. Here is where we didn't do as well 

4. This is what we're doing about that (Don't say this unless 
you mean it!) 
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This is not to suggest that the information officer ought to 
fabricate some public relations gossamer to satisfy the need of 
number four. This is one of those instances when, if the school 
administration doesn't perceive the need to take that step, the 
public information officer must use full resources to influence 
policy. He or she might argue this way: If the tests show that kids 
in the third grade at Crenson Elementary are doing poorly in 
reading comprehension, the very least the* school can do is give 
parents reason to hope the situation will improve. 

A corollary message that must be put across is this: release of 
test data is not an end in itself. As part of a larger accountability 
program, it is a component of a comprehensive effort to upgrade 
schools because the results give direction to many plans for the 
future. There is a need to say this early (assuming that it's true, 
which it was in Maryland). People are going to send letters to the 
"editor" (and educators and legislators) complaining that the 
publication of test results is a sterile exercise that costs a lot of 
money and doesn't help children. It's helpful to be able to point 
to your earlier statement when these charges pop up. 

One problem encountered in Marylaiia may also bedevil 
others. As local school systems began releasing their data, they 
made it known that this was to be part of a larger report to be 
released later- by the state. Enterprising reporters called the state 
department asking for copies now! They began leaning on the 
department staff in mid -November, when local system reports 
began to appear. (The statewide report was not to be presented to 
the governor, as required by law, until mid-January.) We stood 
behind our decision to report first to the governor. 



The Media 

In our survival of this confrontation, there was a message for 
everyone: How you handle a problem like this depends to a large 
extent on your ongoing relationship with news media representa- 
tives. If you have always been candid and helpful, you can emerge 
from this experience without damage to your long-term relation- 
ship with reporters. A detailed press briefing was scheduled in 
January, and long before that day the state department staff had 
tried to anticipate every conceivable line of inquiry. Two 
university authorities on testing who had nothing to do with the 
program were brought in to offer their independent interpreta- 
tions of the meaning of the test results. 

45 



ERIC 



48 



The department established a hold-for-release-embargo of 36 
hours to avoid having reporters dash off "half-baked and semi- 
digested" stories. Office and home telephone numbers of staff 
people who could answer questions were given to all reporters. 
The result — accurate stories and no embargo violations. 

Reporters criticized the state's failure to provide analytical or 
summary data on a report that ran over 600 pages. This material, 
however, had been omitted deliberately because it would have had 
to include a comparison of school systems, something we wished 
to avoid. But the data for comparison were there, and a few 
reporters went to the trouble of digging out the facts for tabular 
presentations in rank order. The story that went on the wire 
service, however, was not one of these, and most school systems 
were spared public comparison. 

Strategies 

This report appears to have been concerned more with 
strategies than techniques. That is not to say that public 
information techniques and processes should be treated casually. 
The Maryland State Dept. of Education did organize a tight and 
careful calendar of activities. It did develop an internal communi- 
cations system that included superintendents, central office staff, 
and principals, working through the school systems themselves and 
through statewide professional organizations. It did organize 
regional workshops to explain to principals and others: (1) how to 
understand the test data, (2) what the public would want to know; 
and, (3) how to develop generally better information attitudes. It 
did employ to the fullest the available professional resources in the 
state. It did plan in detail how to serve media people, staff, 
parents and other key publics. 

Would the information strategy have been different if the test 
results had been better? Probably not. But in all likelihood the 
department would not have worked so hard to prepare the way for 
final statewide release. It would not have involved so many people. 
It would not have helped develop the heightened interest that was 
generated within locd school systems. It would not have swal- 
lowed so many aspirins. 

In the end, telling the public about assessment results 
is nothing more than good informational policy applied 
with a measure of intelligence, or, failing that, sensitivity. 
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CHAPTER 7. 

Local District Dissemination: 
Preparation, Not Prayer 



The state may help the media understand assessment and may 
even satisfy the information needs of the politicians. And the 
people at the intermediate or regional level may contribute further 
to these efforts. But the accountability war and the battles over 
test scores are won or lost at the local level. 

For the school district, and even the local school itself, 
assessment can be a no-win situation no matter how the scores 
turn out. If they're low, community reaction is predictable 
(What's wrong?). If they're high, the immediate question may be, 
"How come they weren't higher?" In other words, whatever the 
situation, it is very risky to release test scores without first 
preparing the community. 

Some educators have argued forcefully that assessment test 
scores are an insignificant measure of student achievement. (Such 
arguments generally follow the presentation of low scores.) This 
"cop-out" approach to assessment just doesn't work. As far as the 
"outside world" is concerned, tests wouldn't be used if they didn't 
mean anything. So, on the premise that the scores are important 
to everybody, begin your dissemination efforts by analyzing your 
publics and understanding what the scores mean to each of them: 

• The students — it's their scores. 

• The parents — it's their kids. 

• The taxpayers — it's their money. 

• The staff - it's their jobs. 

And what about the media? Ordinarily, they are considered a 
prime "audience" for a message, but in reporting assessment scores 
they are critical for another reason: the media serve as a funnel, 
and often a filter, to your other publics. How you approach the 
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media, therefore, depends on what you need to tell your other 
publics. All of this may sound overly complex for public relations 
neophytes, but it need not be. With some intelligent preplanning 
and preparation, the media will work with you and enhance 
communication through other channels. 

In planning dissemination efforts, first consider where the 
various piibHcs will most likely get the word. 

Parents and other citizens will probably hear about it first 
from the media. For the non-parents, this is usually the end of the 
dissemination process. If you lose them here, you may have lost 
them for good. 

Parents, however, have another place to turn. That place is the 
local school. What would be more logical than for the parent who 
has read about test scores in the paper to call or visit the child's 
school for confirmation or explanation? And what would be more 
disastrous to the program than a secretary or teacher who says, 
"The what?'' in response to the inquiry? 

Dissemination Begins at Home 

Therefore, the dissemination plan, like charity, should begin at 
home. Before alerting the media, make sure the staff — and that 
means all school employes — understands what student assessment 
is and what the resulting scores mean. Reporters have been known 
to zero in immediately on a school whose scores were particularly 
high — or low. The principal or teachers who shrug off the 
interviewer with an"Oh, we're not worried about that Mickey 
Mouse program" will undercut whatever communication efforts are 
made by the central office (or the immediate district or the state). 

The staff component of dissemination planning should begin 
months before the tests are given. At that time, all members of the 
staff should be informed about the nature of the educational 
assessment program, what the tests are supposed to show and what 
they will not show, and what will be done with the results. (Don't 
depend on the media coverage the district or state may have 
received when the assessment program was first adopted to do that 
job for you. It's surprising how fast people forget.) Be sure that 
when the scores are released, the staff will know where they came 
from and what they mean. 

When test results become available, another staff, briefing 
should be conducted before scores are presented to the media. 
Think how your faculty would feel if they first read about the 
scores in the newspapers. 
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However, the scores themselves should not be given to staff 
members so early that results will be gossiped about and leaked 
out before you're ready for the media. The task is to develop a 
-..4imeline'-that provides a logical dissemination sequence and gets 
the test data in the hands of staff as rapidly as possible. After all, 
the purpose of assessment is to provide information that wUl help 
teachers improve instruction. This basic fact alone should dictate 
placing teachers early in the dissemination timeline. 

Following the staff briefings, each principal can begin individ- 
ual efforts to inform the school community. Then the central 
office can call a news conference and put the district newsletter to 
press. Ideally, it can all happen within a week. Utopia would be 
for each school to make its report to its community coincide with 
the district-wide release, because the real credibility lies at the 
local school level. 

Once you understand your publics and youVe drawn up the 
dissemination plans, it's time to think about what, and how, 
you're going to communicate. 

First, the Staff 

As soon as the scores are available, the central office should 
hold a detaUed briefing for principals. This should include an 
explanation of the entire program and a thorough analysis of the 

r^ssults, complete with charts and tables. The principals become, in 

effect, the first line of PR, because they must go back to their 
schools to conduct detaUed briefings of their own. So before they 
leave, they should be well-versed in the facts as well as the PR 
"line" the district wants to take. There's a great responsibility here 
for the person who conducts the briefing to make sure the 
principals understand how important a proper attitude is to the 
success or failure of the entire program. Some wUl grasp it quickly; 
others will have to be persuaded. 

Now it's the principals' turn to take the explanations, the 
charts, and the peptalks back to thek schools and conduct similar 

. sessions for their staffs. And this doesn't mean just the faculty, 
either. The secretary who answers the phone and the custodian 
who chats with his neighbor in the barber shop are also part of the 
school family. They cannot be ignored. 

They should not be expected, of course, to master all the 
educational implications of the scores, but they should have a 
general picture and understand the significance and purpose of 
assessment. Remember, to neighbors and casual acquaintances, 
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they work for the school, and consequently their opinions carry a 
lot of authority in their social groups. The principal, therefore, 
should give the entire school staff a general briefing, outlining the 
major implications so that if any employe is confronted by a 
citizen in the supermarket, he will be able to respond confidently, 

*Yes, weve already made plans to beef up our reading in- 
struction, but generally we were pretty pleased with the 
results. "(Note that when they're involved in a program, it's 
''we." When they're left out, more often than not, it's • 
"they. ") 

The teachers, however, need and should want a deeper 
understanding of the testing program and its ramifications to 
prepare them for the parents who want to know what all this 
means to their children. If the scores are below average, does this 
mean theirs is a "bad" school? How do their children's scores 
compare with the school's or the ditrict's? Once again, the 
teacher can destroy the credibiHty of the entire program if he or 
she responds, "Oh, we don't pay any attention to that sort of 
thing" or "Test scores don't mean anything." To parents, test 
scores mean a lot. They represent concrete evidence. They "show" 
whether kids are learning or not learning. Downplaying their 
importance will only sound like an effort to evade responsibility, 
and it will backfire. 

Finally, preferably in a give-and-take session, the faculty 
should come up with a plan of action based directly on the scores. 
Thus, when a parent asks, "What do you have to say about those 
low reading scores?" the answer won't be, "But look how high the 
math scores were!" 

If the scores are low, everybody shouldn't sit around wringing 
their hands, but should think in terms of what can be done to 
bring them up. If they're high, the public relations problem is 
considerably lessened, but this should not be a cue to take bows. 
Staff response, rather, should be in terms of what they intend to 
do to keep them high. 



Now, the Media 

Now that the entire staff is fully informed, it's time to give 
the story to the media. A news conference is the best vehicle. 
Actually, it can be held just about the time the school staffs are 
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being briefed. Since the chances are the material is too complex 
for the pre3s to digest and write comprehensive, accurate stories 
for the next day's paper or that night's broadcast, you can, and 
should, put an embargo on the release. As long as it's no more 
than two or three days, the press will both honor and appreciate 
it. (If the embargo is too long, you may have a problem with 
leaks.) This way you can assure that the story will appear in the 
papers just a day or two after your staffs have been prepared for 
whatever reaction the press reports may stimulate. 

Even if its scores are low, your school district can come out of 
a news conference with its head high if you follow a few basic 
rules: 

• Give the media everything you have. Don't hide anything. 
They'll probably find out anyway, and then you'll be in 
real trouble. 

• Don't behttle or downgrade any of the data. 

• Prepare charts or tables to hand out. Without distorting 
the data, you may be able to arrange the statistics in a 
more easily understood format. 

• Don't let the reporters get you on the defensive. Once they 
do, it's downhill the rest of the way. 

To illustrate the last point, imagine a news conference where 
the reporters are handed a volume of test scores, are given five 
minutes or so to scan it, and then are invited to ask questions. 

"Tell me. Dr. So-and-so," asks a reporter who has managed to 
spot an interesting figure, "what do you intend to do about those 
horrible reading scores in the fifth grade?" 

At that point, Dr. So-and-so must launch a long discourse to 
explain that those scores aren't really "horrible," but it all comes 
out sounding like rationalizing. 

"We've known for a long time," he adds gamely, "that there 
was a problem in reading achievement in the fifth grade." 

"Oh," parries a reporter as he and his colleagues move in. 
"Why didn't you do anything about it back then?" 

If the doctor is articulate, he may end up getting across all the 
points he intended to. But he has lost control of the situation. The 
ideal news conference from the superintendent's point of view is 
one in which he controls the mood, the attitude and the direction 
the questions take. And he can do it by understanding the needs 
of the reporters. 
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Thus the administrator, who is schooled in the ways of media, 
has a brief summary prepared and hands it out to the reporters, 
even before he gives them the test reports. He does not read from 
the summary. He reviews it verbally. 

"On the whole," he may say, "we are very pleased with the 
results of the assessment program. The report does, however, point 
up some weaknesses, such as in fifth-grade reading and seventh- 
grade arithmetic. Believe me, ladies and gentlemen, as soon as we 
received these results we went right to work in these areas. We've 
got specialists working in the elementary and junior high schools 
to get to the roots of these problems and solve them." 

In 30 seconds, the superintendent has taken all the ammuni- 
tion out of the reporters' guns and set the stage for a positive 
approach. The school system, it turns out, is taking all this in 
stride, has the situation well in hand, and is pleased to share it 
with the worid. A simple remark that the report contains no 
surprises will deflate a reporter looking to puff up the story. And 
that really is the goal of the district's public information effort: to 
make sure everybody takes the report seriously. 

Please note it was the superintendent who took charge of the 
news conference, not the public relations officer or testing 
director. The superintendent can and should defer to' the experts 
when the questions get technical, but the superintendent is the 
spokesman the media want. If he/she is not there, and even if the 
other officials acquitted themselves admirably, the media will still 
want comments from the superintendent. 

After this initial press briefing, the superintendent should 
identify a single spofcespi^rson on assessment. This individual 
should be charged with quickly and accurately responding to 
assessment-related questions or inquiries and should be prepared 
to answer the telephone at home at any hour. 

The initial splash will come from the media. 

Radio^and television, of coufse, will boil it down to a minute 
or two at the most, probably ignoring all of the qualifications and 
warnings carefully emphasized in the briefing. But that's the 
nature of the medium. On the other hand, this "incomplete" 
coverage won't do harm if you have reached key audiences and 
opinion leaders earlier. 

The newspapers can serve to fill in the blanks left by the 
electronic media. You probably had ample opportunity to clarify 
things further as reporters labored over their stories during the 
embargo period. They should be encouraged to keep calling if 
there is still something they don't understand. Some districts, in 
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Montgomery County Public Schools 



FOR RELEASE: OCTOBER 2, 1974 P.M. 



COUNTYWIDE TEST SCORES JIELEASED 

Montgomery County public school students scored well above the national 
average on every standardized aptitude and achievement test administered last 
year, according to countywide test results released this week. 

Composite scores for students given the IOWA Test of Basic Skills in 
grades 3, 5, 7, and 9 ranged from 10 to 21 percentile points abov^ the national 



norm, according to the Annual Test Report, which goes to the Board of Education 



Median scores for 11th grade students, who were given the Tests for 
Academic Progress, ranged from 57 in literature to 71 in mathematics. The 
national norm for all tests is 50. 

'In grades 3 and 5. the composite score (generally considered the most 



test were pretty much the same as last year. 

Achievement scores in Grade 3 ranged from 57 on reference materials to 
a high of 73 on punctuation and a composite score of 71. Fifth grade achieve- 
ment scores spread between 59 on math problem solving and 65 on English usage, 
graphs and tables, and math concepts. The fifth grade composite score was 
64. 

At the junior high school level, composite scores decreased from 68 
to 60 in Grade 7 and from 65 to 62 in Grade 9. Some of this decline was 
attributed by school officials to a change in the testing time required by the 
state accountability law. Another state requirement that may have influenced 
scores was giving tests for the first time to many special education students. 

Grade 7 achievement scores ranged from 53 in punctuation to 62 in graphs 
and tables. Ninth grade achievement scores went from a 54 in spelling to a 
high of 68 in map reading— all still well above the national average of 50. 
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reliable measure of over-all student achievement) and theVlscores on each sub- 
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fact, work with the press for months to make sure every aspect of 
the program is clear. 



Enter the Public 

Now you have to brace for the impact of the news on your 
external publics. 

As noted earlier, at this stage the parents take their questions 
to the school staff. They may even ask their children, so a smart 
principal won't overlook that audience either: 

Think how much a parent's mind would be eased if 12-year- 
old Jimmy would come home and say, "We got our test scores 
back today and our teacher said we did pretty well except in 
arithmetic so we worked on a lot of problems and I think I 
understand it better." Scratch one worried mother. 

For the most part, however, the school will feel the impact of 
parent concerns, while the central office will get it from the rest of 
the community and perhaps even from local government officials. 
And it won't take long to find out how well you did in your 
public relations planning. If you did a good job, expect things to 
be quiet. But if you didn't, the phone may ring off the hook. 

Since the release of test scores as part of an accountability 
program is a continuing effort, it is advisable to keep close tabs on 
the reaction to all dissemination activities. That way, patterns of 
questions or|misunderstandings can be identified to give you an 
idea of what loopholes need plugging. And don't be casual about 
it. Keep logs of telephone calls with a notation of the questions 
raised, and keep a letter file. Canvass the schools for reports of the 
reactions there. And, of course, clip the newspapers and sum- 
marize the radio and TV reports. 

If you left a big hole in your information effort, it will show. 
Information weaknesses will surface in the form of the same 
misunderstandings again and again, giving you strong clues to what 
you should concentrate on next year. 

Of course, it's advisable to do follow-up reporting^^ on 
assessment. Some apprehensions about test results can be accom- 
modated weeks after the initial release of information. That's why 
wise disseminators work with their building principals to distribute 
follow-up material showing what is being done with assessment 
results to improve school programs. It is always appropriate to 
show the pubUc — and particularly parents — that you are working 
to improve their child's education. 
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In School Today 
We Got on the 
Scales . . . 



9f 



We took some tests to see "how 
well we are growing" educationally. 

These tests were begun by the Michi- 
gan State Board of Education to see 
how well children are progressing in 
school. 




Educational assessment tests have 
been offered to about 320,000 Michi- 
gan 4th grade and 7th grade students 
every year since 1970. 

The test results are sent back to 
each local school district within a few 
weeks. The information is available for 
each student, classroom, school, and 
district. 

Local educators can study the re- 
sults to se*^e if the students have learn- 
ed certain basic reading and mathe- 
matics skills— that is, if certain edu- 
cational "needs" exist. Then, they can 
use the results with parents to help 
decide on the best educational program 
for the students. 



READING - MATHEMATICS 




This educational "needs assess- 
ment" is one part of a cooperative 
effort between the state, local school 
districts, and citizens to search for 
ways to improve the quality of edu- 
cation for all of Michigan's school 
children. 



Michigan 
State Board of Education 

Where Needs Assessment 
Is An Important Step In Accountability 



The above is an announcement sent home with each stu- 
dent taking part in the Michigan State Dept. of Education 
program. 
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CHAPTER 8. 

Assessment Scores: 
The Principal's Problem 



As the first line of pixblic information and relations at the 
individual school level, the principal has to be ready — and willing 
— to take an active role. The media coverage resulting from the 
central office information effort, except in very small districts, 
will concentrate on systemwide results. Individual schools with 
unusually high - or low - scores may find themselves bombarded 
by reporters, but the majority of the buildings will be generally 
uncovered except for a cryptic column of numbers in a statistical 
summary. The parents of the children who attend those schools, 
however, are no less concerned, so it's up to each principal to 
answer their questions in a satisfactory way. 

A school-based information program is a must. It should be 
aimed at parents - those who read the district-wide results in the 
papers and wonder how their schools fared, and at those who 
don't read the papers at all. The latter group is larger than is 
generally acknowledged, and it is the most susceptible to 
back-fence gossip. In either case, the information is received most 
favorably when it comes voluntarily from the school, not in 
response to an angry protest. 

The wise superintendent will not send his principals out into 
the wilderness without some sort of survival training. The topic 
should be high on the agenda of a principals' meeting at least 
several weeks before the release date, and the superintendent and 
testing director should spend as much^ime as necessary explaining 
what all the scores mean, and how and to which audiences the 
district will explain them. 

The session should be "divided into three distinct parts. First, 
the superintendent should set the stage by stressing the impor- 
tance of the report to the school system and insisting that all 
principals - no matter what the nature of individual results, take 
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them seriously. The superintendent should also state cleariy and 
forcefully what the overall district information poHcy is going to 
be — whether we are pleased, heartened, concerned, and so forth 
- and what steps are being taken at the district level to correct the 
weaknesses uncovered by the tests. 

Next the testing director should explain (with plenty of 
handouts and AV support) what the tests showed and what the 
various numbers mean. This part of the program may be technical, 
but the principals need a thorough understanding of the entire 
assessment program. They need to know what the tests prove and 
do not prove, and how their individual schools fit into the district 
and state picture. Ideally, the school-by-school scores should have 
been distributed to the principals a few days earUer to provide 
background for this meeting. If this is not possible, distribute 
results that day, but near the end of the meeting so principals 
don't study them and compare notes with their neighbors while 
ignoring the speakers. 

The third part of the meeting should be devoted to the public 
information aspects of the assessment report. It should be 
conducted by the public information specialist if the district has 
one, a public information consultant if the district can afford one, 
or someone with some familiarity with the pubUc information 
process. This is a critical phase of the meeting, because it is a fact 
of life that many educators are totally obUvious to the PR 
implications of such a report. Som^ of the principals will grasp the 
situation readily. Others will have to be sold on the importance of 
a local school information effort. And they'll all need some help in 
formulating their individual plans and in phrasing their messages. 

The person conducting this part of the meeting would do well 
to have prepared a Test Score Survival Kit. (See below) 

1 . A sample of the district's proposed news release. 

2. A clear, concise explanation of the entire assessment 
program, including why it was started, what it shows, 
and what will be done with the results. It would 
probably be helpful if it were prepared in question- 
and-answer form, including those questions the general 
public will be most Hkely to ask. The answers should 
be written so the public can understand them, avoiding 
technical language and "educationese." 

3. For those who will communicate their results in 
writing, such as in a community newsletter, a sample 
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narrative to illustrate the writing style and vocabulary 
that will make the program comprehensible to the 
reader. 

Principals should have time to plan and polish their informa- 
tion program, to meet with staff members to explain the test 
results, and to outUne everyone's role in the program. And 
principals ought to give serious thought to turning the release of 
test scores into a full-fledged public relations/information 
campaign. 

An assessment program is, in fact, an ideal vehicle for 
establishing an accountability dialogue with the community. 
Educators may argue that a single Hsting of test results doesn't 
really tell much about an educational program, but they will have 
a difficult time convincing the community. So why not take 
advantage of the requirement to issue a report by incorporating it 
into a broader information package that tells where you've been, 
where you are, and where you hope to go. 

Ideally, this should evolve into an annual effort, because any 
report on school objectives should center on the year's progress 
toward attaining them. It sounds like a big project, and it is. But 
since it will require the involvement of the entire faculty in setting 
the objectives for the school, it will pay off instructionally as well 
.as in good public relations. 

The Montgomery County (Md.) Board of Education mandated 
in 1974 that each of the district's 202 schools issue annual school 
progress reports each October. Rather than leave the contents to 
each principal's whim, the board issued detailed guidehnes. The 
first go-around was very successful. PTA's devoted entire meetings 
to discussing the reports, and many parents used them to back up 
their requests at budget hearings. 

On the whole, the board felt that the program came very close 
in only one year to achieving its primary purpose: informing 
parents of the needs and strengths of their school, and involving 
them in planning to meet the unique objectives of that school. The 
program was renewed for 1975 with only minor changes. 

Each report is to be divided into five major sections: 

1. Community - a description of the characteristics of the 
attendance area of the school, and a statement of the 
major community concerns as expressed by the PTA, 
advisory groups, letters and phone calls and surveys. ^ 
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2. Enrollment — pertinent statistics showing trends and 
characteristics of the student body. Here is where the 
results of the standardized tests are reported, including an 
emphasis on those areas where the school scored a certain 
amount above, or below, the norm. This section can also 
include information on tests administered by the school, 
performance on college admission tests, results of any 
follow-up study that may have been done on graduates, 
and so forth. 

3. Staff - the number of full-time staff, both professional 
and supporting, their years of experience and level of 
training, and their assignments in the type of organiza- 
tional plan used by the school. 

4. Building — a description of the school plant itself — the 
capacity, number of rooms, special facilities, etc., with a 
description of how the building design affects the organiza- 
tion or program. Immediate plans for renovations or 
additions should be reviewed. 

5. Objectives — for the past year, a list of the objectives and a 
discussion of how well the school did in attaining them; 
for the current year, a description of each objective, the 
strategy for attaining it, and the evidence that will indicate 
success. The guidelines caution principals that the setting 
of objectives requires the involvement of staff and commu- 
nity, and that each school objective should be related to 
the information in the previous sections of the report. 

As a part of an annual report, test scores are more than just an 
isolated column of numbers. They can become an important step 
in a dynamic process of setting objectives, involving parents, and 
assuring accountability. And that, of course, was the reason for 
most assessment programs in the first place. 

The central office in Montgomery County prepared a detailed 
set of guidelines for each principal, including a prototype report — 
actually the report of Brookview Elementary School, which was 
submitted early and rewritten by the system's director of 
information. 

One of the most useful sections of this 16-page sample report 
was a discussion of Brookvie\v's test scores in simple, everyday 
language that parents could understand. Although Brookview's 
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scores were, on the whole, very high, the language in the narrative 
still serves as an excellent example of style for all schools, no 
matter what their standing. The following pages feature excerpts 
from that report. 

Brookview Elementary School 
Montgomery Co., Md. 
Annual Report 1974 

During April-May, third- and fifth-year students 
in all county schools are given two batteries of tests. 
The CAT (Cognitive Abilities Test) gives an indication 
of our youngsters' learning aptitudes and the ITBS 
(Iowa Tests of Basic Skills) provides a measure of 
academic progress in certain skills. While standardized 
tests measure only a small part of all the learning that 
occurs in school, they are important indicators of 
individual pupil progress, as well as the success of 
some aspects of the school and school system's 
educational program. 

Each fall, Brookview teachers study the test data 
to pinpoint the strengths and weaknesses of each 
child. It is important information to use for setting 
objectives and planning instruction in the coming 
year. Each fall, also, the Test Report to Parents, 
which shows each child's test scores, is mailed to 
parents. Teachers interpret these reports at a confer- 
ence with the parents. 

How Did Brookview Do? 

A table on p. 61 shows Brookview's aptitude and 
achievement scores in percentiles. The table compares 
the verbal aptitude percentile of the CAT and the 
composite achievement percentile of the ITBS. The 
fact that these two percentile scores are the same 
indicates that the third graders are learning just about 
as their abilities would lead us to expect. 

All the third grade scores are very good, except 
for reading comprehension. The score here is 10 
points below the verbal aptitude. On the other hand, 
in relation to aptitude, this score is 8 percentile 
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Brookview Elementary School 
Test Information 

Median National Percentile Scores 

1973-74 





Grade 3 


Grade 5 


School 


Comparable 
MCPS Schools 


School 


Comparable 
MCPS Schools 


Scholastic Aptitude 










Verbal 


88 


87 


71 


71 


Quantitative 


97 




67 




Nonverbal 


81 




65 




Achievement 










Vocabulary 


84 


82 


65 


65 


Reading Comprehension 


78 


78 


62 


60 


Spelling 


90 


83 


54 


62 


Capitalization 


88 


81 


55 


62 


Punctuation 


87 


84 


54 


61 


Language Usage 


80 


80 


57 


63 


Map Reading 


87 


83 


58 


63 


Reading Graphs & 










Tables 


88 


81 


57 


65 


Knowledge and Use of 










Reference Materials 


87 


76 


66 


64 


Mathematics Concepts 


87 


. 79 


67 


67 


Mathematics Problem 










Solving 


85 


79 


55 


59 


Composite Achievement 


88 


86 


62 


67 
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points higher than it was last year, and the vocabulary 
score is 16 points better than last year. This suggests 
that our efforts to improve our language arts program 
have been effective, and we will be continuing our 
focus on objectives to develop better comprehension 
skills. 

The test results at the fifth-year level do not look 
as good. These youngsters are not doing as well as we 
expect them to do. Spelling and punctuation are the 
weakest areas, with capitalization and usuage close 
behind. Map reading and use of graphs and tables 
need concentrated attention as well as math problem 
skills. There is an improvement of 7 percentile points 
in reading comprehension since last year. Vocabulary 
remains about the same. We have our work cut out 
for us in terms of upper level program improvement. 

To improve these programs, the staff worked 
during the teacher orientation days in August to 
identify the underachievers. We used test score data 
to pinpoint these students and consulted with the 
teachers they had last year. We began identifying the 
specific skill areas in which these children need help 
and began setting objectives for them for this year. 
Further assessment, diagnosis, and objective-setting 
will continue throughout the year. 

An important thing to remember is that third- 
year achievement test results are not just a measure of 
what was taught in the third year — they are 
indicators of the sum total of your child's learning 
through his third year of school. The same is true for 
fifth-year students. So it takes more than a year for 
the impact of program improvements to show up in 
test results. 

What Do the Numbers on 
the Table Mean? 

Tests were given last school year to all pupils in 
the grade(s) indicated. These tests were scored ac- 
cording to instructions from the test publisher. Each 
number on the table represents the midpoint of the 
scores of all youngsters at this grade level in the 
school. For example, a score of 60 means that half 
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the pupils got scores higher than 60, while half got 
lower scores. 

Each numbei is expressed as a "percentile," 
which permits comparison of the pupils in this school 
with pupils at the same grade level in schools across 
the United States. The national midpoint on any test, 
by definition, is the 50th percentile — meaning that 
half the students across the nation get scores higher 
than 50, while half get lower. If the school percentile 
score is higher than 50, it means, the students are 
doing better than the national average; if the number 
is lower than 50, they aren't doing as well. A school 
percentile score of 80 would mean that half the 
pupils at that grade in the school got scores higher 
than 80 percent of their classmates across the nation. 

The objectives that follow this discussion relate 
directly to the needs pointed up by the test scores. 
One goal, for example, is "to improve reading 
comprehension with specific emphasis on its applica- 
ir tion to mathematics problem solving." This will be 

done, it goes on, through the use by each child of a 
card file containing mathematics terms, and other 
strategies. 

"We looked at our teaching— the reading and 
interpreting of maps, tables, graphs, and pictures-and 
were not satisfied," the report went on. "Test results 
confirm our concern. Therefore, we wrote a third 
objective, as follows: To improve skills of reading and 
interpreting maps, tables, graphs and pictures." 

This illustrates how the test scores can become an 
integral part of the entire instructional effort in the 
school rather than just a miscellaneous piece of 
information that relates to nothing. The idea of 
issuing annual school reports was a good idea before 
assessment. It's a better idea now. 
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CHAPTER 9. 

One Last TTme 



If nothing else, the preceding pages should have demonstrated 
that telling the public about assessment results does not have to be 
a traumatic experience. When handled properly, the dissemination 
of test results can provide an excellent opportunity to build a 
bridge between the schools and the community. 

Experienced practitioners of school public relations have 
probably recognized the approach advocated here as no more and 
no less than sensible application of basic PR techniques. And one 
doesn't have to be schooled in the PR specialty to see the need for 
dissemination founded on a policy of openness and supported 
with large does of common sense and sensitivity. 

One purpose of any educational accountability program 
should be to help the schools become more responsive to the 
needs of their publics. And this is impossible unless those publics 
know, and understand, how the entire education "family" is 
working to improve education. The component of accountability 
known as assessment is vital to both educational and public 
understanding. 

When those involved with assessment programs realize that 
assessment dissemination can be a cornerstone in building under- 
standing, they wUl have taken a large step along the road to the 
ultimate objective - doing a better job of helping kids learn. 
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